Lifi-powered content-aware large-scale data processing factory

ABSTRACT

Systems, methods and apparatus are provided for a reusable, client-server based ecosystem designed to support content-aware, LiFi-powered transfer of large-scale, semi-structured data files. Containerized client-side applications may include a LiFi communication engine (LCE), a job control engine (JCE), and an execution hub that is configured to interface with the JCE, the LCE, job stakeholders and downstream applications. A central server may include a server-side LCE configured for two-way communication with the client-side LCE. Each LCE may be configured to cluster semi-structured data into data packets, broadcast data packets using an LED array, receive data packets using an array of photoreceptors and synchronize received data packets.

FIELD OF TECHNOLOGY

Aspects of the disclosure relate to LiFi-powered transmission oflarge-scale, semi-structured data files, such as XML files.

BACKGROUND OF THE DISCLOSURE

Data transmission of large XML-format files presents a range oftechnical challenges. XML stands for Extensible Markup Language. An XMLfile uses a tag-based language to create a structured summary of a feedsource. The combination of structured and unstructured data oftenresults in transmission errors and file corruption. The size of thefiles increases the likelihood of error and limits the speed of thetransfer. Slow transfer times may lead to repeated timeouts and renderthe transfer more vulnerable to attack.

Conventional file transmission methods include various forms of wiredconnective infrastructure or wireless transfer via wireless fidelity(WiFi). WiFi transmission uses radio frequency to induce a voltage in anantenna to transmit data.

Light Fidelity (LiFi) technology offers a number of advantages fortransfer of large files. LiFi is a light-based communication systemcapable of transmitting data wirelessly at high speed using lightemitting diodes (LEDs). LiFi transmission speeds may be more than onehundred times faster than conventional WiFi. LiFi infrastructure is alsorelatively simple, in contrast with the radio transmitters and antennaerequired for WiFi communications.

The speed of LiFi transfer may enable real-time parallel processing oflarge-scale files, vastly improving processing efficiency. The speed ofLiFi transmission may also limit data leakage and thus protect againstadversarial attacks during the data transmission process.

It would be desirable to provide a client-server ecosystem thatintegrates LiFi protocols for transmission of large semi-structured datafiles. It would be desirable to improve reusability by incorporatingcontainerized client-side applications that could be deployed across arange of platforms. It would be desirable for the client-sideapplications to be content-aware in order to reduce transmission errorsand further improve efficiency.

SUMMARY OF THE DISCLOSURE

Systems, methods, and apparatus for a LiFi-powered, content-aware,large-scale data processing factory are provided.

A client-side LiFi communication engine (LCE) may be configured fortwo-way client-server communication. The client-side LCE may packetize asemi-structured data file according to LiFi protocols and assign arequest identifier. The client-side LCE may broadcast packets ofsemi-structured data to a server-side LCE via LiFi using an LED array.The client-side LCE may receive processed data from a server-side LCEvia LiFi using an array of photoreceptors. The client-side LCE maysynchronize the received processed data using the request identifier.

The client-side container may include a job control engine (JCE). TheJCE may receive a job associated with a semi-structured data file andgenerate file metadata associated with the job.

The client-side container may include an execution hub configured tointerface with the JCE and the LCE. The execution hub may prioritize ajob received from the JCE according to the associated metadata.

The client-side container may include quality gates. A first qualitygate may validate the size and format of the semi-structured data forcompliance with LiFi transmission protocols. A second quality gate mayvalidate the content of the semi-structured data for compliance withLiFi transmission protocols. The second quality gate may extract andvalidate XML header and trailer nodes.

A central server may perform processing operations on thesemi-structured data. The central server may include a server-side LCEconfigured for two-way communication. The server-side LCE may packetizethe processed data according to LiFi protocols. The server-side LCE maybroadcast packets of processed data to a client-side LCE via LiFi usingan LED array. The server-side LCE may receive semi-structured data froma client-side LCE via LiFi using an array of photoreceptors. Theserver-side LCE may synchronize the received semi-structured data usingthe request identifier.

BRIEF DESCRIPTION OF THE DRAWINGS

The objects and advantages of the disclosure will be apparent uponconsideration of the following detailed description, taken inconjunction with the accompanying drawings, in which like referencecharacters refer to like parts throughout, and in which:

FIG. 1 shows illustrative apparatus in accordance with principles of thedisclosure;

FIG. 2 shows illustrative apparatus in accordance with principles of thedisclosure;

FIG. 3 shows illustrative system architecture in accordance withprinciples of the disclosure;

FIG. 4A shows an illustrative process flow in accordance with principlesof the disclosure;

FIG. 4B shows an illustrative process flow in accordance with principlesof the disclosure; and

FIG. 5 shows illustrative system architecture in accordance withprinciples of the disclosure.

DETAILED DESCRIPTION

Systems, methods and apparatus for a LiFi-powered, content-aware,large-scale data processing factory are provided.

Conventional wireless networking relies on wireless fidelity (WiFi)technology. WiFi transmits data using the radio frequency spectrum.

Light Fidelity (LiFi) is a two-way network protocol for high speedconnectivity using light technology. LiFi may capture data in modulatedlight frequencies. The driver-circuit in LED bulbs may encode andtransmit data by switching the LED on and off at rates so fast that theflickering is indiscernible to the human eye. The data may be decoded byan optical sensor on the receiving end and converted back into anelectronic data stream. In some embodiments, LED bulbs may be dimmed tolevels below human visibility while still emitting enough light to carrydata.

LiFi technology presents a number of advantages over conventional WiFi.One advantage is transmission speed. LiFi transfers may occur at speeds100 times faster than conventional WiFi. Another advantage is capacity.WiFi relies on the radio-frequency spectrum which is subject tocongestion and slowing due to high traffic. LiFi, on the other hand,uses the visible light spectrum which is 10,000 times larger than theradio bandwidths and is therefore not limited by spectrum capacity.While WiFi frequencies may eventually become crowded and slower whenlarge amounts of data are transferred, LiFi is able to deliverconsistent large data transfers. Additionally, unlike WiFi, LiFifrequencies may be used in electromagnetic sensitive areas withoutcausing interference.

LiFi infrastructure is also relatively simple, in contrast with theradio transmitters and antennae required for WiFi communications. Thelimited hardware required for LiFi communications also improves energyefficiency.

LiFi technology may provide solutions for processing of largesemi-structured data files. Conventionally, real-time processing of XMLfiles greater than typical RAM size has not been practical. A reusableclient-server based ecosystem designed to support content-aware LiFiwould enable centralized processing of these files.

Centralized processing may reduce overall processing time, latency andcold starts. For large-scale data, a single memory cache may not be ableto accommodate the data. Shifting to distributed or parallel processingwould require cold start time, as the system would need to perform loadbalancing for the initial setup before processing could even begin. TheLiFi-powered, client-server based ecosystem reduces latency by usingpacketization and LiFi transmission along with server-based parallelprocessing to improve processing time.

In some embodiments, the speed of LiFi transfer may allow batchprocessing to be converted into real-time online processing. The speedof LiFi transmission also limits data leakage and therefore protectsagainst adversarial attacks during the data transmission process.

Conventionally, the combination of structured and unstructured data inXML files often results in transmission errors and file corruption. LiFitransmission may be combined with content-aware operations thatpreprocess and validate the data to further reduce the risk of filecorruption during transmission. Data integrity validation along withsubsequent LiFi packetization and synchronization may ensure thathigh-speed LiFi transmission is integrated with additional safeguardsfor secure transfer.

For the sake of illustration, the invention will be described as beingperformed by a “system.” The system may include one or more features ofapparatus and methods that are described herein and/or any othersuitable device or approach.

The system may involve client-server architecture. The system mayinclude one or more client computers. Each client computer may be incommunication with a central server. By enabling data transmission forlarge XML files, LiFi allows for centralized processing at the serverside. Centralized processing improves efficiency and enables the re-useof processed data across different architectures.

A client computer may include one or more client-side containers. Acontainer typically encompasses an entire runtime environment.Containerized software may be packaged along with its libraries anddependencies into a single unit. Containers virtualize CPU, memory,storage, and network resources at the OS level, allowing container-basedapplications to run consistently across many different target platforms.With the incorporation of containerized client-side applications, theclient-server based ecosystem may be reusable, for content-aware LiFitransmission across a range of platforms.

The client-side container may include a job control engine (JCE). TheJCE may be the primary point of entry for the data factory ecosystem.The JCE may receive a job request. The job request may involvelarge-scale, semi-structed data files. The files may be XML files.Illustrative job requests may involve data preprocessing such as datacleaning or data transformation or statistical insights such as dataaggregation.

The JCE may generate metadata for the job request. Illustrative metadatamay include the job objective, the source of the file, and thedestination location. The JCE may add metadata indicating priority ofthe job or the relationship between different parts of a compound job.

The client-side container may include an execution hub. The executionhub may orchestrate job execution. The execution hub may receiveincoming job requests from the JCE. The execution hub may interface witha LiFi communication engine (LCE) for data transfer as well as withdownstream applications that will make use of the data.

The execution hub may prioritize jobs according to the metadataassociated with each job request. The execution hub may monitor jobstatus and update job execution information in an enterprise database.The execution hub may include an exception queue for jobs that are notfulfilled because they do not comply with LiFi transmission parameters.The execution hub may include a message queue service for notifyingstakeholders or downstream applications regarding job status. Theexecution hub may reconcile processed XML data received from the serverwith the original job requests. The execution hub may generate reportsand dashboard information. Illustrative reports may include dailyexecution trends or metrics evaluating communications.

The JCE may transmit a list of job requests to the execution hub. Theexecution hub may prioritize the requests based on the metadatagenerated by the JCE.

The client-side container may include multiple quality gates to ensurethat both the format and content of the data is suitable for LiFitransmission. A first quality gate may validate file formatrequirements. In order to be suitable for LiFi transfer, the data may berequired to have a predefined format and a predefined size. For example,LiFi resources may be reserved for files in XML format of 16 GB orlarger. The first quality gate may filter out files that do not meetsystem requirements.

A second quality gate may validate the content of an XML file. Thesecond quality gate may include a processing queue. The second qualitygate may validate XML header and trailer nodes for data integrity. Thesystem may check for anomalies or corrupted data using any suitable dataquality metric. Extraction of header and trailer nodes is a quick andsimple way to access XML file content for validation. Illustrativeheader and trailer data may include a number of records, a sum offinancial amounts, or any suitable summary information.

If the data fails validation at either of the quality gates, theexecution hub may store the job in the exception queue. The executionhub may report exceptions to the JCE or to one or more stakeholders.

The client-side container may include a LiFi communication engine (LCE).If the quality gates validate the XML data, the XML data may proceed tothe LCE. The LCE may be configured for two-way data communication usingLiFi technology.

The LCE may include a packetization component. The packetizationcomponent may divide the XML data into data packets for LiFitransmission. The packets may be of equal size or may be broken upaccording to any suitable method. During packetization, XML data may beclustered based on the job priorities, job type, or business unit wherethe job originated. For example, a job request may involve hundreds ofXML files, each larger than 16 GB. The LCE may use clustering to groupthe files based on a process identified in the file name, file size, orany sequence-based XML processing within the job.

The LCE may include a broadcasting component. The broadcasting componentmay control the transmission of data packets using LiFi technology. TheLCE may use light intensity or light frequency modulation to encode thedata. The LCE may use an array of LEDs, micro-LEDs, or any suitablelight source to transmit the data. The invention may also be compatiblewith light sources emitting light outside the visible spectrum usinginfrared or ultraviolet frequencies. The LEDs may be switched on and offat rates indiscernible to the human eye. The LEDs may be dimmed tolevels below human perception.

The LCE may include a receptor component. The receptors may receive atransmission of the data packets using LiFi technology. The LCEreceptors may include an array of photoreceptors capable of detectingthe light transmission and converting it to an electronic data stream.

The LCE may include a synchronization component. The synchronizationcomponent may confirm the completeness and accuracy of packettransmission. Prior to transmission, the XML data may be assigned aunique request identifier. The LCE may match received data received withthis request identifier. Synchronizing received data with the requestidentifier ensures that all of the packetized data associated with aparticular job is reassembled following transmission.

The client-side LCE may communicate with a server-side LCE via the LiFibroadcasting component. The server-side LCE may include the samepacketizing, broadcasting, receiving and synchronizing components as theclient-side LCE. Server-side LCE receptors may receive the files fromthe client-side LCE. Server-side LCE synchronization may validate thecompleteness and accuracy of the transmission using the requestidentifier.

The server may process the data using a range of business logicapplications. For example, the server may provide structure and analysisthrough ETL, data warehousing, data mining, data quality checks or dataanalysis.

The server may transmit the processed data to one or more client-sidecontainers via LiFi using the server-side LCE. Server-side LCEpacketization may cluster the processed data into data packets andserver-side LCE broadcasting may transmit the data packets. The abilityto transmit large files using an LCE enables the system to consolidateprocessing at the server level, improving both processing efficiency andreusability of the processed data.

Client-side LCE receptors may receive processed data from theserver-side LCE. Client-side LCE synchronization may validate thecompleteness and accuracy of the transmission by matching the processedXML data with the request identifier. Exceptions may be transmitted toan exception handler in communication with the execution hub.

Processed data that is successfully transmitted from the server may besent from the LCE directly to the execution hub. The execution hub mayreconcile the processed XML data with the job request to ensure thatprocessed data correlates with the job objectives received from at theJCE. While the LCE synchronizes processed data at the XML file level,the execution hub reconciles the data at the job level. The executionhub message queue may store information about completed jobs. Themessage service may interact with downstream client applications thatwill make use of the processed data.

The system may be applied to mainframe jobs. Mainframe jobs are similarto distributed jobs but run from mainframe servers. In some embodiments,the client container may be adapted to interface with mainframe O/Sbased servers in place of distributed servers.

One or more non-transitory computer-readable media storingcomputer-executable instructions are provided. When executed by aprocessor on a computer system, the instructions perform a method forcentralized processing of large-scale, semi-structured data files.

The method may include, at a client container, receiving a job requestassociated with a semi-structured data file and generating file metadataassociated with the job. Containerized client-side applications maypacketize the semi-structured data by clustering the data based at leastin part on the metadata associated with the job. The containerizedclient-side applications may broadcast the packets of semi-structureddata to a server platform via LiFi using an LED array.

The method may include, at a server platform, receiving the packets ofsemi-structured data using an array of photoreceptors and synchronizingthe received packets. The method may include processing the data at theserver platform. The method may include using server-side applicationsto packetize the processed data and broadcast the packets of processeddata to the client-side applications via LiFi.

The method may include, at the client container, receiving the packetsof processed of processed data from the server platform using an arrayof photoreceptors and synchronizing the received packets. The method mayinclude reconciling the received processed data with the job request.

Systems, methods, and apparatus in accordance with this disclosure willnow be described in connection with the figures, which form a parthereof. The figures show illustrative features of apparatus and methodsteps in accordance with the principles of this disclosure. It is to beunderstood that other embodiments may be utilized, and that structural,functional and procedural modifications may be made without departingfrom the scope and spirit of the present disclosure.

The steps of methods may be performed in an order other than the ordershown and/or described herein. Method embodiments may omit steps shownand/or described in connection with illustrative methods. Methodembodiments may include steps that are neither shown nor described inconnection with illustrative methods. Illustrative method steps may becombined. For example, an illustrative method may include steps shown inconnection with any other illustrative method.

Apparatus may omit features shown and/or described in connection withillustrative apparatus. Apparatus embodiments may include features thatare neither shown nor described in connection with illustrativeapparatus. Features of illustrative apparatus may be combined. Forexample, an illustrative apparatus embodiment may include features shownor described in connection with another illustrative apparatus/methodembodiment.

FIG. 1 is a block diagram that illustrates a computing device 101(alternatively referred to herein as a “server or computer”) that may beused in accordance with the principles of the invention. The computerserver 101 may have a processor 103 for controlling overall operation ofthe server and its associated components, including RAM 105, ROM 107,input/output (“I/O”) module 109, and memory 115.

I/O module 109 may include a microphone, keypad, touchscreen and/orstylus through which a user of device 101 may provide input, and mayalso include one or more of a speaker for providing audio output and avideo display device for providing textual, audiovisual and/or graphicaloutput. Software may be stored within memory 115 and/or other storage(not shown) to provide instructions to processor 103 for enabling server101 to perform various functions. For example, memory 115 may storesoftware used by server 101, such as an operating system 117,application programs 119, and an associated database.

Alternatively, some or all of computer executable instructions of server101 may be embodied in hardware or firmware (not shown).

Server 101 may operate in a networked environment supporting connectionsto one or more remote computers, such as terminals 141 and 151.Terminals 141 and 151 may be personal computers or servers that includemany or all of the elements described above relative to server 101. Thenetwork connections depicted in FIG. 1 include a local area network(LAN) 125 and a wide area network (WAN) 129, but may also include othernetworks.

When used in a LAN networking environment, computer 101 is connected toLAN 125 through a network interface or adapter 113.

When used in a WAN networking environment, server 101 may include amodem 127 or other means for establishing communications over WAN 129,such as Internet 131.

It will be appreciated that the network connections shown areillustrative and other means of establishing a communications linkbetween the computers may be used. The existence of any of variouswell-known protocols such as TCP/IP, Ethernet, FTP, HTTP and the like ispresumed, and the system may be operated in a client-serverconfiguration to permit a user to retrieve web pages from a web-basedserver. Any of various conventional web browsers may be used to displayand manipulate data on web pages.

Additionally, application program 119, which may be used by server 101,may include computer executable instructions for invoking userfunctionality related to communication, such as email, short messageservice (SMS), authentication services and voice input and speechrecognition applications.

Computing device 101 and/or terminals 141 or 151 may also be mobileterminals including various other components, such as a battery,speaker, and antennas (not shown). Terminal 151 and/or terminal 141 maybe portable devices such as a laptop, tablet, smartphone or any othersuitable device for receiving, storing, transmitting and/or displayingrelevant information.

Any information described above in connection with database 111, and anyother suitable information, may be stored in memory 115. One or more ofapplications 119 may include one or more algorithms that encryptinformation, process received executable instructions, interact withenterprise systems, perform power management routines or other suitabletasks. Algorithms may be used to perform the functions of one or more ofgenerating job metadata, validating file data, identifying exceptions,clustering XML data into data packets, broadcasting data packets,receiving data packets, synchronizing data packets, reconciling data,and/or perform any other suitable tasks.

The invention may be operational with numerous other general purpose orspecial purpose computing system environments or configurations.Examples of well-known computing systems, environments, and/orconfigurations that may be suitable for use with the invention include,but are not limited to, personal computers, server computers, hand-heldor laptop devices, tablets, mobile phones and/or other personal digitalassistants (“PDAs”), multiprocessor systems, microprocessor-basedsystems, set top boxes, programmable consumer electronics, network PCs,minicomputers, mainframe computers, distributed computing environmentsthat include any of the above systems or devices, and the like.

The invention may be described in the general context ofcomputer-executable instructions, such as program modules, beingexecuted by a computer. Generally, program modules include routines,programs, objects, components, data structures, etc. that performparticular tasks or implement particular abstract data types. Theinvention may also be practiced in distributed computing environmentswhere tasks are performed by remote processing devices that are linkedthrough a communications network. In a distributed computingenvironment, program modules may be located in both local and remotecomputer storage media including memory storage devices.

FIG. 2 shows an illustrative apparatus 200 that may be configured inaccordance with the principles of the invention.

Apparatus 200 may be a computing machine. Apparatus 200 may include oneor more features of the apparatus that is shown in FIG. 1.

Apparatus 200 may include chip module 202, which may include one or moreintegrated circuits, and which may include logic configured to performany other suitable logical operations.

Apparatus 200 may include one or more of the following components: I/Ocircuitry 204, which may include a transmitter device and a receiverdevice and may interface with fiber optic cable, coaxial cable,telephone lines, wireless devices, PHY layer hardware, a keypad/displaycontrol device or any other suitable encoded media or devices;peripheral devices 206, which may include counter timers, real-timetimers, power-on reset generators or any other suitable peripheraldevices; logical processing device 208, which may generate job metadata,validate file data, packetize data according to LiFi transmissionprotocols, and perform other methods described herein; andmachine-readable memory 210.

Machine-readable memory 210 may be configured to store inmachine-readable data structures: job data, job metadata, job updates,exception data, request identifiers, data packet information, and anyother suitable information or data structures.

Components 202, 204, 206, 208 and 210 may be coupled together by asystem bus or other interconnections 212 and may be present on one ormore circuit boards such as 220. In some embodiments, the components maybe integrated into a single chip. The chip may be silicon-based.

FIG. 3 shows high-level system architecture 300. Architecture 300 showsaspects of the reusable client-server based, LiFi-powered ecosystem. Theclient-side portions of the LiFi-powered ecosystem may be containerizedand deployed across a variety of client platforms. Client containers302, 303 and 304 may each include a client-side LCE. The client-sideLCEs may each be configured for two-way communication with a server-sideLCE on server platform 306. This arrangement improves efficiency byusing LiFi transmission to enable centralized processing of large-scalesemi-structured data files.

FIGS. 4A and 4B show aspects of illustrative process flow 400. Processflow 400 illustrates the flow of information through the client-serverbased, LiFi-powered ecosystem. Process flow 400 may include one or morefeatures shown in architecture 300.

FIG. 4A focuses on the client-side flow of information. At step 402, ajob control engine receives a job associated with a semi-structured datafile and generates file metadata associated with the job. At step 404,the job is transmitted to the execution hub. The execution hub maymanage the job by assigning priority, sequestering exceptions,reconciling received data with assigned jobs, communicating withapplications via a message queue, and generating reports and dashboardupdates. At step 406, the file may be validated by one or more qualitygates. The quality gates may confirm that file size and format areconsistent with LiFi transmission protocols. The quality gates mayvalidate the content of the file to minimize corruption during transfer.At step 408, exceptions may be transmitted to the execution hub andstored in an exception queue. The execution hub may communicate theexception to the job control engine using the message queue. Theexecution hub may generate a report regarding the exception.

At step 410, a client-side LCE may cluster the semi-structured data intoa set of data packets. The client-side LCE may broadcast the datapackets according to LiFi protocols using an array of LED bulbs. At step412, the server-side LCE receives the data packets from the client-sideLCE. Server-side LCE photoreceptors may convert the transmission to anelectronic data stream for processing. Server-side LCE synchronizationmay ensure the transmission is complete.

FIG. 4B includes additional elements of process flow 400, showingtransmission of processed data from the server-side LCE to the clientenvironment. The data received from the server is shown using a brokenline. Steps 402 to 412 of FIG. 4B are the same steps shown in FIG. 4A.

The server platform may perform a range of operations on thesemi-structured data received from the client-side LCE. At step 412, theserver-side LCE packetizes the processed data and broadcasts it to theclient-side LCE using LiFi. The data packets may be received byclient-side LED photoreceptors and synchronized to ensure thetransmission is complete. At step 414 exceptions may be identified. Ifthere is no exception, the processed data may transmitted directly tothe execution hub. The execution hub may reconcile the processed datawith job objective information. The execution hub may post informationabout the completed job on the message queue. If an exception isidentified, the processed data may be transmitted to exception handler416. Exception handler 416 may communicate the data to the executionhub.

FIG. 5 shows illustrative system architecture 500 for a reusableclient-server based, LiFi-powered ecosystem. Elements of systemarchitecture 500 may correspond to elements shown in system architecture300 and process flow 400.

Client container 502 may be deployed on a client computer. Job controlengine 504 may receive a job associated with a semi-structured datafile. job control engine 504 may transmit the job to Exception Hub 506.

Exception Hub 506 may prioritize the job using the job prioritizer.Exception Hub 506 may transmit the file to quality gate 508. Qualitygate 508 may filter files based on file size and file format. Qualitygate 510 may validate the content of the file using any appropriate datametric. Files that fail validation may be transmitted back to theexecution hub and stored in the exception queue.

Files validated by quality gate 510 be transmitted to client-side LCE512. Client-side LCE 512 may packetize the semi-structured data andbroadcast the data packets using LiFi technology.

Server platform 516 may provide centralized processing for data receivedfrom multiple client-side LCEs. Server-side LCE 518 may receive the datapackets from client-side LCE 512 using an array of photoreceptors.

Server platform 516 may perform processing operations on the receiveddata. Server-side LCE 518 may cluster the processed data into datapackets and broadcast the processed data to client-side LCE 512 viaLiFi.

Client-side LCE 512 may receive the processed data using LiFi receptors.Client-side LCE 512 may synchronize the processed data at the XML levelusing a request identifier. Client-side LCE 512 may transmit exceptionsto exception handler 514.

Client-side LCE 512 may transmit synchronized processed data toexecution hub 506. Execution hub 506 may reconcile the processed dataagainst the job request. Execution hub 506 may update job status orupdate one or more databases. Execution hub 506 may use the messagequeue to communicate job completion to a stakeholder or a downstreamapplication. Execution hub 506 may transmit processed data to one ormore downstream applications.

Thus, methods and apparatus for a LIFI-POWERED CONTENT-AWARE LARGE-SCALEDATA PROCESSING FACTORY are provided. Persons skilled in the art willappreciate that the present invention can be practiced by other than thedescribed embodiments, which are presented for purposes of illustrationrather than of limitation, and that the present invention is limitedonly by the claims that follow.

What is claimed is:
 1. A system for centralized processing oflarge-scale semi-structured data files via light fidelity (LiFi)transmission, the system comprising: a client-side environmentcomprising a client computer and a client container, the clientcontainer comprising a client-side LiFi communication engine (LCE)configured for two-way client-server communication, the client-side LCEconfigured to: packetize a semi-structured data file according to a setof LiFi protocols and assign a request identifier; broadcast the packetsof semi-structured data to a server-side LCE via LiFi using an LED arrayto transmit binary data; receive processed data from a server-side LCEvia LiFi using an array of photoreceptors; and synchronize the processeddata using the request identifier; and a central server configured toperform one or more processing operations on the file data, the centralserver comprising a server-side LCE configured for two-waycommunication, the server-side LCE configured to: receive packets ofsemi-structed data from a client-side LCE via LiFi using an array ofphotoreceptors; synchronize the received packets using the requestidentifier; packetize processed data according to a set of LiFiprotocols; and broadcast packets of processed data to the client-sideLCE via LiFi using an LED array to convey binary data.
 2. The system ofclaim 1, wherein the client container further comprises a job controlengine configured to receive a job associated with the semi-structureddata file and generate file metadata associated with the job.
 3. Thesystem of claim 2, wherein the client container further comprises anexecution hub configured to interface with the job control engine andthe LCE, the execution hub configured to prioritize a plurality of jobsbased at least in part on the metadata associated with each job.
 4. Thesystem of claim 3, wherein the execution hub is further configured toreconcile processed data received by the client-side LCE with a jobobjective received from the job control engine.
 5. The system of claim1, wherein the client container further comprises a quality gateconfigured to validate the size and format of the semi-structured datafile for compatibility with one or more LiFi transmission parameters. 6.The system of claim 5, wherein the quality gate is a first quality gate,and the system further comprises a second quality gate configured tovalidate content of the semi-structured data file for compatibility withone or more LiFi transmission parameters.
 7. The system of claim 6,wherein the validation comprises extracting and validating XML headersand trailers.
 8. The system of claim 2, wherein packetizing thesemi-structured data file comprises clustering the data based at leastin part on the metadata associated with the job.
 9. The system of claim1, wherein the semi-structured data file is an XML file with a sizegreater than 16 gigabytes.
 10. One or more non-transitorycomputer-readable media storing computer-executable instructions which,when executed by a processor on a computer system, perform a method forcentralized processing of large-scale, semi-structured data files vialight fidelity (Lifi) transmission, the method comprising: at a clientcomputer comprising a client container and one or more containerizedclient-side applications: receiving a job associated with asemi-structured data file and generating file metadata associated withthe job; prioritizing the job according to the metadata associated withthe job; validating content of the semi-structured data file forcompatibility with one or more LiFi transmission parameters; packetizingthe semi-structured data according to a set of LiFi protocols andassigning a request identifier to the data; broadcasting the packets ofsemi-structured data to a server platform via LiFi using an LED array;receiving processed data from the server platform via LiFi using anarray of photoreceptors; and synchronizing the processed data receivedfrom the server platform using the request identifier; and at the serverplatform: receiving the packets of semi-structured data via LiFi fromthe containerized client-side applications using an array ofphotoreceptors; synchronizing the received packets using the requestidentifier; performing one or more processing operations on the receiveddata; packetizing the processed data according to a set of LiFiprotocols; and broadcasting the packets of processed data to thecontainerized client-side applications via LiFi using an LED array. 11.The media of claim 10, further comprising reconciling processed datareceived by the containerized client-side applications with a jobobjective.
 12. The media of claim 10, wherein validating the content ofthe semi-structured data file comprises extracting and validating XMLheaders and trailers.
 13. The media of claim 10, further comprisingvalidating the size and format of the semi-structured data file forcompatibility with one or more LiFi transmission parameters.
 14. Themedia of claim 10, wherein packetizing the semi-structured datacomprises clustering the data based at least in part on the metadataassociated with the job.
 15. The media of claim 10, wherein thesemi-structured data file is an XML file with a size greater than 16gigabytes.
 16. A method for centralized processing of large-scalesemi-structured data files via light fidelity (Lifi) transmission, themethod comprising: at a client computer comprising a client containerand one or more containerized client-side applications: receiving a jobassociated with a semi-structured data file and generating file metadataassociated with the job; prioritizing the job according to the metadataassociated with the job; validating content of the semi-structured datafile for compatibility with one or more LiFi transmission parameters;packetizing the semi-structured data according to a set of LiFiprotocols and assigning a request identifier to the file; broadcastingthe packets of semi-structured data to a server platform via LiFi usingan LED array; receiving the processed data from the server platform viaLiFi using an array of photoreceptors; and synchronizing the processeddata received from the server platform using the request identifier; andat the server platform: receiving the packets of semi-structed data viaLiFi from the containerized client-side applications using an array ofphotoreceptors; synchronizing the received packets using the requestidentifier; performing one or more processing operations on the receiveddata; packetizing the processed data according to a set of LiFiprotocols; and broadcasting the packets of processed to thecontainerized client-side applications via LiFi using an LED array. 17.The method of claim 16, further comprising reconciling processed datareceived by the client-side containerized applications with a jobobjective.
 18. The method of claim 16, wherein validating the content ofthe semi-structured data file comprises extracting and validating XMLheaders and trailers.
 19. The method of claim 16, further comprisingvalidating the size and format of the semi-structured data file forcompatibility with one or more LiFi transmission parameters.
 20. Themethod of claim 16, wherein packetizing the semi-structured datacomprises clustering the data based at least in part on the metadataassociated with the job.
 21. The method of claim 16, wherein thesemi-structured data file is an XML file with a size greater than 16gigabytes.